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ABSTRACT 

This research's objective was to identify the terminal efficiency of the Massive Online Open 
Course "EducationalInnovation with Open Resources "offered by a Mexican private university. 
A quantitative methodology was used, combining descriptive statistics and probabilistic 
models to analyze the levels of retention, completion, and desertion, as well as the 
characteristics of the students who completed the course. The results show a 14% of student 
retention and an 11.7% of student completion, relative to the total number of participants, 
who had some common characteristics: having a graduate (master or doctorate), being 
experienced in online education, committed to the course and self-taught. The participants 
who abandoned the course expressed the following reasons: problems with the course's 
structure, limitations in the use of information and communication technologies or limited 
English proficiency, family reasons or low time disposition. It is recommended to take actions 
that will increase the knowledge in order to explain the MOOCs' desertion rates and to 
strengthen their structures to improve the retention and completion rates. 

Keywords: Distance education. Open educational resources, MOOC, Terminal efficiency. School 
desertion 


INTRODUCTION 

The evolution of distance education and technological advances signify an important 
opportunity to increase education's access and contribution to the compliance of international 
commitments regarding education. In this respect, the United Nations Educational, Scientific, 
and Cultural Organization (UNESCO, 2002, 2012) has established that free access to 
educational resources is a strategy to upgrade the quality of education, to facilitate the 
dialogue about policies, to interchange knowledge and to develop skills. 

Massive Online Open Courses (MOOC) are an emerging practice in open learning. It began in 
2008, when George Siemens and Stephen Downes offered the course Connectivism and 
Connective Knowledge in the University of Manithoba in Canada; the course had a duration of 
12 weeks and 2,300 students enrolled (Fini, 2009; Wiley & Hilton III, 2009). 

Among their characteristics, MOOCs allow the construction of bonds between hundreds or 
even thousands of students who self-organize their participation, learning goals, knowledge. 
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abilities and interests (McAuley, Stewart, Siemens & Cormier, 2010). Additionally, their free 
online access enables the enrollment of a large number of students (SCOPEO, 2013). 

Currently, Coursera, EdX, and Udacity are among the platforms that host MOOCs. For Latin 
American participants, MiriadaX and RedunX are available (Lushnikova, Chintakayala, 
Rodante, 2013; SCOPEO, 2013). Many Latin American universities have developed their own 
MOOCs in their institutional platforms or even through social networking sites such as 
Facebook. Nevertheless, only three Latin American universities are part of the Coursera 
community. 

In developing countries, the use of MOOCs is an alternative educational offering for 
professionals who look for complementary training and education. In addition, these courses 
allow the acquisition of new knowledge and skills in fields that could provide them the 
opportunity for a better income or to continue learning throughout life. Massive courses attract 
thousands of participants who are interested in the offered topic; however, it is important to 
note that approximately only 10% of the registered students complete the course (Lushnikova 
et al., 2013). Such case is reported by the University of Toronto (Harrison, 2013), where the 
percentage of terminal efficiency varied between 3% and 16%. On a similar note, SCOPEO 
(2013) reported an average of 13.5% of students who completed the course, out of 188,802 
registered participants in 58 courses from 18 universities. 

The low terminal efficiency rates of MOOCs reveal a lack of self-regulation and self-motivation 
in students (Lushnikova et al., 2013). Likewise, Clow (2013) remarks that the student's 
compromise level may diminish as the courses move forward. On this respect, Siemens and 
Tittenberg (2009) established that desertion rates could be minimized by providing more 
attention to the students regarding the components of effective learning, motivation, 
institutional support and free access to educational resources, in order to promote the 
development of interpersonal relationships among peers, faculty, and teaching staff. 

A dropout in online courses refers to the persons who inconspicuously discontinue their 
participation in the course; however, they are different from passive participants, students 
who do not unregister and continue the course without active collaboration in it (Rodriguez, 
2012). Additionally, the author signals that the desertion rates and passive behavior of some 
MOOCs' participants are among the educators' concerns. 

In Latin America, experiences related to MOOCs are still developing; due to the growing 
research demand for these educational practices, it is necessary to identify and overcome the 
difficulties and obstacles in order to increase their dissemination and implementation, as well 
as the promotion of this type of initiatives, so they can be integrated in the public agenda of 
countries, institutions and inter-institutional projects in Latin America (Mortera, 2012). These 
circumstances have been confirmed by Liyanagunawardena, Adams and Williams (2013), who 
signal a lack of research and information about MOOCs that can explain why participants do 
not complete a course. 

According to Gomez-Zermeno (2012), it is important to create a context that supports 
innovative practices to evaluate the results of the undertaken efforts when new ways of 
teaching and learning are promoted. Given the research demands about the potentialities of 
MOOCs, it is important to inquire into the terminal efficiency of an MOOC offered in Spanish. 
This investigation generated information about the MOOC Educational Innovation through 
Open Resources, offered in Coursera on September of 2013. The study had the following 
research questions: What was the terminal efficiency of participants in the course Educational 
Innovation through Open Resources? What are the characteristics of the participants who 
successfully completed the course? With a quantitative methodology that combines 
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descriptive and econometric statistics, the research results show the retention and terminal 
efficiency rates of the course, the features of students who completed the course, as well as 
the causes of dropout and abandonment. A probabilistic model was used to identify the weight 
of each one of the dropout factors and thus evaluate the terminal efficiency of the MOOC 
"Educational Innovation through Open Resources". 

RESEARCH METHOD 

Based on the research questions, the study opted for a quantitative methodology, combining 
the use of descriptive and econometric statistics, which allowed to identify the MOOCs' 
participants profile and to calculate the dropout rates and terminal efficiency. Probabilistic 
models were used to identify the weight of each one of the dropout factors. The dependent 
variable is a binary variable (if the student abandoned the course, it was assigned the value of 
1 and 0 on the contrary) and the independent or explanatory variables include factors such as 
gender, age, previous experience in virtual education and electronic media, educational level, 
English proficiency, and intrinsic characteristics such as being proactive, innovative and self- 
taught. 

The quantitative approach was used to analyze information through statistical methods; 
participants answered a diagnostic and final survey to inform about their opinions and 
perspective, this allowed understanding better the numerical data. Recognizing the 
participants' experiences allowed the comprehension of the phenomenon (Aleman & Gomez- 
Zermeno, 2012; Gomez-Zermeno, Rodriguez Arroyo & Marquez Guzman, 2013). 

From an explanatory point of view (Creswell & Plano, 2011), the study sought to understand 
why the dropout phenomenon occurs and under what conditions, in order to identify the 
reasons why MOOC's participants decide to dropout and not complete the course. We used a 
non-experimental, cross-section and ex-post-facto design, and the participants' information 
was collected during August-September of 2013 (Valenzuela y Flores, 2012). 

MOOCS's Description and Context 

The MOOC "Educational Innovation through Open Resources", offered in Coursera, can be 
catalogued as continuous training; although it's access was not restricted, it was designed 
mainly for basic education teachers in Mexico. Two Head Professors who have ample 
experience in the design of online courses, the use of Open Education Resources and 
Educational Technology designed it. The context of the MOOC's creation is within a prominent 
Mexican University, leader in Educational Technology and the first Mexican University to 
impart courses via satellite in the mid 90's. This University has offered online courses through 
their virtual campus for over 20 years. 

The course covers the subject of the selection, use and reuse of open educational resources, 
the possibilities the repositories that house these materials have, search strategies and 
integration into educational processes, as well as measuring and assessing their impact on 
learning of the participants. Thus, the participant would develop digital skills and instructional 
design skills to integrate open educational resources (OER) in their learning environments 
through open educational practices. 

In four modules, participants were able to watch videos or read about the course's topics, and 
interacted with other participant in discussion boards by answering opinion questions. Digital 
portfolios and self and peer evaluations were used to assess the students' understanding of 
how to integrate OER in institutional and learning processes in their own learning 
environment. 
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Research Population and Sample 

According to the statistics provided by the platform, 20,400 people registered for the course, 
which started in September of 2013. From the initial population, 4,407 participants completed 
the instrument called, "Pre-diagnosis survey" and 3,547 people answered the "Initial survey." 
The data of each participant was given a unique identification code, which cannot be 
duplicated; by combining these two registers, we obtained 5,854 participants who were 
considered as the study sample. Table 1 shows the results of both surveys. 


Table: 1 

Results from Pre-diagnostic and Initial Survey 


Instrument 

Participants 

% 

Pre-diagnostic survey 

2,307 

39.4 

Initial survey 

1,447 

24.7 

Both instruments 

2,100 

35.9 

Participants of the research population 

5,854 

100.0 


Considering the information of 5,854 out of 20,400 people who registered, it was important to 
confirm if the number of existing cases permitted a statistical analysis in order to identify 
significant differences among the constructs. The formula to calculate the sample of finite 
populations was used; in Social Sciences research, the maximum sampling error is 5%, and 
although this data was not available, 5,854 of the participants who completed either the pre¬ 
diagnosis or the initial survey correspond to a sampling error of 1.2%, considered statistically 
significant. 

Instruments 

The data collection instruments were created by the head professors of the MOOC; there 
following 6 surveys were applied and used for collecting data: 

> Pre-diagnosis survey: is a structured questionnaire with 49 questions, combining 
closed questions, multiple options and weighted options using Likert rating scale. 
This instrument collects information about: general student data, MOOCs 
perceptions, skills and knowledge of information technologies, use of search 
engines, use of OER, Innovation and open education movement, 

> Initial survey: collects general information on participants such as age, sex, marital 
status, country of residence, education, experience in online education, computer 
and internet connection type, social networks used, among others. Additionally, 
survey asked about the reasons for registering in the course, the level of 
commitment and hours per week to devote to the course. 

> Topic 1 self-assessment: 15 multiple choice questions regarding the topic 1 Open 
educational movement. 

> Topic 2 self-assessment: 15 multiple choice questions regarding the topic 2 Search 
of educational resources. 

> Topic 3 self-assessment: 15 multiple choice questions regarding the topic 3 Use of 
open educational resources in learning processes. 

> Topic 4 self-assessment: 15 multiple choice questions regarding the topic 4 
Mobilization of open educational resources in learning environments. 

These instruments do not have psychometric test results and do not correspond to an 
evaluation research design, but rather an exploratory design. They were completed online by 
participants using the platform Coursera. 
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Research Procedure 

During the first week of the course, the pre-diagnosis and initial electronic surveys were 
administrated; a link was provided thru the platform, and e-mail notification was sent to all 
the registered participants. The instruments were completed virtually and voluntarily; the 
results show that more than 70% of the enrolled students did not answer the surveys. 

The self-assessment instruments for topics 1 to 4 were provided at the end of each week thru 
a direct link. As the course moved forward, the number of participants who completed the 
instruments decreased. Table 2 shows that at the end of the first week, 30% of the participants 
delivered evidence of their work, which diminished to a 15.6% in the last week of the course. 


Table: 2 

Level of participation throughout the course 


Instrument 

Participants 

% 

Research population 

5854 

100.0% 

Topic 1 Self-assessment 

1779 

30.4% 

Topic 2 Self-assessment 

1165 

19.9% 

Topic 3 Self-assessment 

967 

16.5% 

Topic 4 Self-assessment 

911 

15.6% 


RESULTS ANALYSIS 

Using a quantitative method, in this research, descriptive and econometric statistics were used 
to calculate the retention and terminal efficiency rates of the course, to identify the features 
of students who completed the course, as well as the causes of dropout and abandonment. 
Finally, a probabilistic model was created to identify the weight of each one of the dropout 
factors and thus evaluate the terminal efficiency of the MOOC "Educational Innovation through 
Open Resources". 

Retention and Terminal Efficiency Rates 

The databases were processed in SPSS (Statistical Package for the Social Sciences) and STATA, 
in order to calculate the MOOCs' retention rate by dividing the number of participants who 
completed the last self-assessment (Topic 4) by the total number of students who fulfilled the 
first instruments (pre-diagnosis and initial survey). 

Table 3 shows the retention and terminal efficiency results of the MOOC, regarding the gender 
and total participants. 14.5% of retention was reported, meaning 818 participants who 
remained engaged until the last week of the course; while 11.7%, 683 participants, delivered 
all four topic's self-assessments. 


Table: 3 

Terminal efficiency according to gender 


Efficiency 

Female 

Male 

Total percentage 

Total 

number 

Desertion 

82.2% 

82.4% 

86.0% 

5,036 

Retention 

17.8% 

17.6% 

14.0% 

818 

Incomplete 

85.5% 

84.3% 

88.3% 

5,171 

Completed 

14.5% 

15.7% 

11.7% 

683 


It is interesting, that when comparing the retention results (14%) and terminal efficiency 
(11.7%) of the course, the percentages are similar to those denoted by the University of 
Toronto which reported 8% average of terminal efficiency (Harrison, 2013), 10% reported by 
Miriadax (SCOPEO, 2013), and 10% reported by Lushnikova et al. (2013). 
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Features of Students who Completed the Course 

Researchers carried out a comparative analysis of the characteristics of the persons who 
abandoned the course against those who remained, and the features confrontation of those 
who completed the course and those who did not deliver the self-evaluation assessments. This 
analysis shows that the students who possess a master's degree or higher are more likely to 
remain in the course, rather than those who only have a professional degree or lower 
educational level (Table 4). The persons who did not have previous experience with online 
education had lower rates of completion and terminal efficiency (Figure 1). 


Table: 4 

MOOCs Terminal efficiency according to educational level 


Educational Level 

Student's retention 

Terminal efficiency 

High school 

11.4% 

10.7% 

Technical career 

9.0% 

8.0% 

Undergraduate degree 

14.4% 

12.1% 

Master's degree 

19.0% 

15.9% 

Doctorate 

20.1% 

17.2% 

Post-doctorate 

28.2% 

23.1% 


I 

Retention Terminal efficiency 

■ Experienced in online education 

■ Unexperienced in online education 


Figure: 1 

Terminal efficiency in the MOOC according to experience with online courses. 

Additionally, results show that the participants who expressed more initial commitment 
presented higher completion and terminal efficiency rates; these participants planned to 
complete all the activities and evaluations to obtain a diploma and registered to the course as 
a way to complement their previous studies. On the contrary, the people who registered out 
of curiosity or who were not committed to the activities registered lower completion and 
terminal efficiency rates. These results confirm the statements by Cabrol & Szekely (2012), 
regarding the importance of relevant education as a strategy to avoid academic failure and 
desertion. Likewise, Aleman, Sancho-Vinuesa y Gomez-Zermeno (2015) highlight the need for 
resource and strategy analysis under selection criteria. 

Individuals who expressed to have economic stability, either full-time or part-time workers, 
business owners, at home work or people who have a flexible schedule, reported higher 
completion and terminal efficiency rates, as opposed to those who study in high school or a 
undergraduate degree. 

It is important to mention that the participants who described themselves as self-learners 
obtained higher completion and terminal efficiency rates. 
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Regarding the features of the participants with higher completion rates, they displayed a 
proficient use of information technologies, of digital resources design, intermediate English 
level, knowledge organization skills, participation in research networks, and other 
characteristics. In relation to the people who described themselves as pro-active, there were 
no significant differences in the results of completion and terminal efficiency. 

On the contrary, the people without knowledge about copyrights, web information 
administration, use of OER in the classroom, and lack of experience in research networks 
presented higher abandonment of the course. 

It must be noted that the previously mentioned characteristics have significant statistic 
differences, which were validated by the Pearson's chi-squared test, at 95% confidence 
interval. 

Causes of Dropout and Abandonment 

Coursera politics do not allow e-mail sharing in order to avoid spam. Therefore, it was not 
possible to administrate a sanctioned follow-up survey to explore the specific reasons some 
participants had when deciding to dropout or abandon the course. As an alternative, a 
reflection about various messages shared on the discussion forums was made. Among the 
stated reasons that may cause the discouragement of the participants and to abandon the 
course were: 

> Difficulties with the structure of the course, and lack of a tutorial to guide users. 

> The quality of the materials was also criticized. 

> Family reasons and no availability for the course. 

> Limitations on the use of information technology or in the English language. 

> Limitations of the Coursera platform. 

Probabilistic Model to Evaluate Terminal Efficiency of MOOC 

Given the results, the terminal efficiency of the MOOC was analyzed through the construction 
of a probabilistic model in order to quantify the weight of the main features of participants 
who do not complete the course. The main results of the models (table 5) are: 

> The odds of desertion of the MOOC increases 5.7% when the participant has an 
undergraduate or lower degree, and the probability rises 5% when the student does 
not have knowledge about copyrights. On the contrary, the odds of desertion 
diminishes 7% if the participant is older than 55 years old, 17% when they show a 
strong commitment to the MOOC and 4.2% when they have a full-time or part-time 
job 

> The odds to complete the MOOC increases by 3.2 % when the participant is female 

and 3.8 % when it has no copyrights knowledge. By contrast, the odds of not 
completing decreases 8% when participants are over 55, 15 % when they have a 
strong commitment, and another 3.2 % when they are excited by applying course's 
knowledge in their practice as teachers or daily life. 
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Table: 5 

Probabilistic model of the participants who do not complete an MOOC 


Traits of the participant 

Desertion 

Dropout 

Female 

0.0187 

0.0323* 

Older than 55 years old 

-0.0698* 

-0.0803** 

Experienced in online education 

-0.02 

-0.015 

Does not have a degree (high school or technician) 

0.0568* 

0.0396 

Full-time or part-time job 

-0.042** 

-0.021 

Plans to accomplish activities and tests to obtain certificate 

-0.1702** 

-0.1499** 

Null (0 - 20%) - IT domain to create audio, video, images, 
etc. 

0.0451 

0.0423 

Null (0 - 20%) - Knowledge about copyrights 

0.0501** 

0.0382* 

Not important - Research filters 

-0.0491 

0.0215 

Null (0 - 20%) I do not know the English language 

0.0271 

0.0245 

Lack of confidence - During information research 

0.0989 

0.1115 

Self-taught (Constantly updating my knowledge) 

-0.0031 

-0.0165 

Null (0 - 20%) - Knowledge of use of techniques and 
methods to organize knowledge in an accessible and 
considering scientific objectives, observable facts, and / 
or measurable 

Null (0 - 20%) - Domain to communicate in virtual 
environments 

0.003 

-0.0026 



-0.0756 

-0.0526 

Null (0 - 20%) - Domain to determine credibility of 
information 

0.0001 

0.042 

To complement my classes (design and/or prepare 
courses) 

-0.0226 

-0.0316* 

Uncertain, because they do not know what students will 
think of the use of open resources 

-0.0085 

-0.0169 

Does not participate in research network 

0.0306 

0.022 

Not willing to participate as facilitator or Teaching 

Assistant 

0.0092 

0.0166 


Note: ** significant at 95 % confidence level * significant at 90 % confidence. 


For probabilistic models, the dependent variable is constrained between zero and one, being 
derived from the cumulative distribution function (Gujarati, 1997). One way of evaluating the 
probabilistic models is derived from the goodness of fit (R 2 ); however, when dealing with 
nonlinear models the goodness of fit is meaningless in terms of the defined coefficient of 
determination. The pseudo R 2 of the model to neglect and no completion corresponds to 0.0249 
and 0.0252, respectively. However, note that by supplementing with another statistic shows 
that the model correctly classifies drops to 75% of cases, whereas the model for not completed 
correctly classified 78% of instances. So the models can be considered quite acceptable for a 
multi factorial phenomenon as desertion and abandonment. 

CONCLUSIONS 

The main results of this research reveal low terminal efficiency rate of a MOOC, which was 
offered by a higher education institution. Although there was a positive response from the 
students, the percentage of participants who successfully completed the course indicates the 
opposite; therefore, it is important to study the reasons that led the participants to enroll and 
the causes of desertion. 
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The MOOC analyzed in this study had a rate of 11.7 % completion rate, which represents the 
number of students who delivered their respective assessments and answered throughout the 
course evaluations. In contrast, the high dropout rate of 86% agrees with the statements of Clow 
(2013), who mentions that the abandonment of online courses is higher than in classroom 
education. This result is parallel with the report of the University of Toronto, which showed a rate 
of approximately 8% of students who completed the course (Harrison, 2013). Similarly, Lushnikova 
et al. (2013) indicate that about 10 % of the students who enrolled in a MOOC managed to complete 
the course. In this connection, Siemens and Tittenberger (2009) note that desertion rates can be 
minimized by upgrading the course components that lead to an increase in the motivation levels 
and better student-faculty relationships. 

This study identified the main characteristics of the participants who managed to stay and complete 
the course, being favored those with graduate degrees, online educational previous experience, 
greater commitment to the course and economic stability. Other found features were the advanced 
or expert proficiency in the use of information technology, advanced proficiency in creating digital 
resources, intermediate English language skills, advanced proficiency in the use of techniques and 
methods to organize knowledge and active participation in research networks, among others. On 
the other hand, those who decided to leave the course indicated problems with the structure and 
guidance in the course, limitations on the use of information technology or in English, in addition 
to the limited availability of time due to family or work reasons. It is noteworthy that among the 
deserters were participants with a high school or bachelor education who do not participate in 
research networks, which is consistent with the statement made by Siemens (2005), who defines 
the research groups as a means to update the knowledge and maintain connections for continuous 
learning. 

The results of the probabilistic models constructed to measure the weight of the characteristics of 
participants who drop out or fail to complete a MOOC course, reflect that the participants' likelihood 
to leave the MOOC course increases when the participant has a lower educational degree level and 
has no knowledge of copyrights. On the contrary, the probability of abandonment decreases when 
participants are over 55, have a strong commitment to the MOOC and when they have full or partial 
employment. In terms of completeness, the chances of not completing the MOOC increases when 
the participant is female and does not have any knowledge of copyrights. By contrast, the odds of 
not completing decreases when participants are over 55 when they have a strong commitment, and 
when they are excited by applying knowledge of the course in their practice as teachers or daily 
life. 

In light of the results obtained in this research, two possible courses of action arise as strategies to 
increase the level of awareness of the terminal performance of MOOCs' participants: First, the 
application of surveys to students who decide to leave the course to obtain additional information 
about their reasons. Second, to increase the level of terminal efficiency of MOOCs, it is proposed to 
include in the course's structure items such as a welcome tutorial to guide novice users in this type 
of courses, on their function and structure; organize discussion forums by language, country or 
thematic affinities, and improve the quality of the videos and captions. 

These recommendations are consistent with the approach of Aguaded (2013) who states the need 
to strengthen areas, such as interaction with the facilitators, collaborative and interactive work, 
respect cultural and linguistic diversity, for MOOC constituting an exceptional learning experience. 
Similarly, for the effectiveness of these courses you must create a scaffold to guide and help the 
participants to achieve their learning goals (Salmeron, Rodriguez, & Gutierrez2010). 

This research enables institutions that offer MOOC courses to consider the characteristics of 
the participants, in order to achieve greater efficiency and lower desertion rates. Thus, the 
educational practices in MOOC will benefit by improving their implementation and continue 
gathering information on new experiences in such resources. 
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